ABSTRACT
BACKGROUND: The COVID Radar app was developed as a population-based surveillance instrument to identify at-risk populations and regions in response to the COVID-19 pandemic. The app boasts of >8.5 million completed questionnaires, with >280,000 unique users. Although the COVID Radar app is a valid tool for population-level surveillance, high user engagement is critical to the success of the COVID Radar app in maintaining validity. OBJECTIVE: This study aimed to identify optimization targets of the COVID Radar app to improve its acceptability, adherence, and inclusiveness. METHODS: The main component of the COVID Radar app is a self-report questionnaire that assesses COVID-19 symptoms and social distancing behaviors. A total of 3 qualitative substudies were conducted. First, 3 semistructured focus group interviews with end users (N=14) of the app were conducted to gather information on user experiences. The output was transcribed and thematically coded using the framework method. Second, a similar qualitative thematic analysis was conducted on 1080 end-user emails. Third, usability testing was conducted in one-on-one sessions with 4 individuals with low literacy levels. RESULTS: All 3 substudies identified optimization targets in terms of design and content. The results of substudy 1 showed that the participants generally evaluated the app positively. They reported the app to be user-friendly and were satisfied with its design and functionalities. Participants' main motivation to use the app was to contribute to science. Participants suggested adding motivational tools to stimulate user engagement. A larger national publicity campaign for the app was considered potentially helpful for increasing the user population. In-app updates informing users about the project and its outputs motivated users to continue using the app. Feedback on the self-report questionnaire, stemming from substudies 1 and 2, mostly concerned the content and phrasing of the questions. Furthermore, the section of the app allowing users to compare their symptoms and behaviors to those of their peers was found to be suboptimal because of difficulties in interpreting the figures presented in the app. Finally, the output of substudy 3 resulted in recommendations primarily related to simplification of the text to render it more accessible and comprehensible for individuals with low literacy levels. CONCLUSIONS: The convenience of app use, enabling personal adjustments of the app experience, and considering motivational factors for continued app use (ie, altruism and collectivism) were found to be crucial to procuring and maintaining a population of active users of the COVID Radar app. Further, there seems to be a need to increase the accessibility of public health tools for individuals with low literacy levels. These results can be used to improve the this and future public health apps and improve the representativeness of their user populations and user engagement, ultimately increasing the validity of the tools.
ABSTRACT
Due to the COVID-19 pandemic, a sudden shift was warranted from face-to-face to digital interviewing. This shift is in line with the existing trend of digitalization. However, limited literature is available on how to conduct focus group interviews online successfully. This research note provides practical guidelines, tips, and considerations for setting up and conducting online synchronous focus groups for eight relevant factors: preparation, the number of participants, the duration, a break, the usability of the online platform, the interaction between participants and researchers, support and roles of the research team, and privacy considerations. These guidelines were formulated based on the available literature and our own positive hands-on experiences. We consider online focus groups to be an excellent option when taking into account the considerations related to the eight factors.
ABSTRACT
BACKGROUND: Monitoring of symptoms and behavior may enable prediction of emerging COVID-19 hotspots. The COVID Radar smartphone app, active in the Netherlands, allows users to self-report symptoms, social distancing behaviors, and COVID-19 status daily. The objective of this study is to describe the validation of the COVID Radar. METHODS: COVID Radar users are asked to complete a daily questionnaire consisting of 20 questions assessing their symptoms, social distancing behavior, and COVID-19 status. We describe the internal and external validation of symptoms, behavior, and both user-reported COVID-19 status and state-reported COVID-19 case numbers. RESULTS: Since April 2nd, 2020, over 6 million observations from over 250,000 users have been collected using the COVID Radar app. Almost 2,000 users reported having tested positive for SARS-CoV-2. Amongst users testing positive for SARS-CoV-2, the proportion of observations reporting symptoms was higher than that of the cohort as a whole in the week prior to a positive SARS-CoV-2 test. Likewise, users who tested positive for SARS-CoV-2 showed above average risk social-distancing behavior. Per-capita user-reported SARS-CoV-2 positive tests closely matched government-reported per-capita case counts in provinces with high user engagement. DISCUSSION: The COVID Radar app allows voluntarily self-reporting of COVID-19 related symptoms and social distancing behaviors. Symptoms and risk behavior increase prior to a positive SARS-CoV-2 test, and user-reported case counts match closely with nationally-reported case counts in regions with high user engagement. These results suggest the COVID Radar may be a valid instrument for future surveillance and potential predictive analytics to identify emerging hotspots.